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(54) Image processing method and apparatus 



(57) A method of autonnatically discriminating the 
document direction is Iriade by paying attention to a 
point that a factor which most correctly expresses the 
document direction is a character A sample character 
image which is extracted from inputted image data is 
character recognized from directions of 0°, 90°, 180°, 
and 270°. In accordance with precision values (reliabil- 
ities) of the character recognition results, the direction 
of the highest precision value is discriminated as a prop- 
er direction. 
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Description 

The invention relates to image processing method 
and apparatus for automatically discriminating the direc- 
tion of a document of an image read by a photoelectric 
converting device such as a CCD or the like 

Hitherto, in an information processing apparatus, 
characters are generally recognized from original infor- 
mation (image data) read by an optical reader such as 
a scanner orthe like, However, the character recognition 
is performed on the assumption that the characters were 
certainly inputted so as to face the proper direction 
(namely, the characters are arranged in the proper di- 
rection for the scanner), so that the character recogni- 
tion result exactly differs from the input original 

For example, in case of characters 1001 shown in 
Figs, 1 0A to 1 CD, when an image is inputted so that the 
character faces the proper direction as shown in Fig„ 
10A, a recognition result as a character 1011 is correctly 
obtained. However, in the case where an image is rotat- 
ed by 270° and is inputted as shown in Fig. 10B, it is 
erroneously recognized as a character 1 01 2 Jn the case 
where image data is rotated by 180° and the image is 
inputted as shown in Fig IOC, it is erroneously recog- 
nized as a character 1013, In the case where image data 
is rotated by 90° and the image is inputted as shown in 
Fig„ 10D, it cannot be recognized, Those recognition re- 
sults are shown for convenience of explanation and the 
actual results don*t always become the results as men- 
tioned above. 

In order to avoid the problem such that when the 
input image is character recognized as it is as men- 
tioned above, so long as the direction of a document is 
not the proper direction, the correct result cannot be ob- 
tained, there is also the following method Namely, the 
image inputted by the scanner is displayed and the user 
judges whether the image has been inputted in the prop- 
er direction or not„ When the inputting direction is not 
the proper direction, the rotation of the image in an im- 
age memory is instructed so as to manually rotate the 
direction of the image by 90°, 180°, and 270° by the us- 
er, thereby rotating the image, orthe setting direction of 
the original to the scanner is changed to the proper di- 
rection and the image is again inputted orthe like, there- 
by correcting the image to the proper direction, and after 
that, the start of the character recognition is instructed. 

However, owing to the improvement of a processing 
speed of the scanner, an automatic document feeding 
function called an auto feeder is annexed to the appa- 
ratus and a plurality of originals can be automatically 
read and inputted„ The number of originals as targets to 
be processed can be set to a large quantity of papers 
nowadays. In such a situation, it is very difficult to per- 
form operations for manually judging the proper direc- 
tion one by one for a plurality of originals and for cor- 
recting the direction of the original image. 

The contents of the document of the original is ex- 
pressed by various patterns such as vertical writing, lat- 



eral writing, and the like. That is, for example, there are 
various kinds of patterns such as document of the A4 
size which is vertically placed and is written in the lateral 
direction (see Fig„ l!A: for example, this pattern is often 
5 used in a lateral writing Japanese document, English 
document, orthe like), document of the A4 size which 
is laterally placed and is written in the lateral direction 
(see Fig, 11 B: for example, such a pattern is frequently 
used in a document having a long sentence, a document 
fo for the OHP, a document in case of reducing and copy- 
ing a document of the A3 size, 84 size, orthe like which 
is laterally placed, or the like), a document of the A4 size 
which is laterally placed and is laterally written and in 
which the column setting is switched at the center (see 
^5 Fig, 11C: for example, such a pattern is often used in 
case of continuously reducing and copying two sheets 
of documents of the A4 size), a document of the A4 size 
which is vertically placed and is vertically written (see 
Fig. 11 D), and the like.. 

On the other hand, the direction of an image which 
is displayed on a display screen is determined by the 
scanning direction (image reading direction) of the scan- 
ner when inputting the image. Therefore, in case of 
reading the original at a position such that the scanning 

^5 direction of the scanner coincides with the direction of 
a character train of the document, the document is dis- 
played at the inherent position. 

However, in case of the scanner of the A4 size, 
since the direction of the original is unconditionally de- 

30 termined, the original cannot be read at the position 
such that the scanning direction of the scanner is made 
coincide with the direction of the character train of the 
document and, for example, as shown in Fig„ 128, the 
document is displayed in a state in which it is rotated by 

35 90° from the inherent position, so that there is a case 
where it is very difficult to read the image 

Hitherto, there is a method of automatically discrim- 
inating the direction of the inputted original so as to re- 
duce the operation to correct the direction of the original 

40 by the operator as mentioned above. Namely, as shown 
in Fig,, 1 3A, there is a method whereby lines of a table, 
a separator (a), and the like are extracted from an input- 
ted original image and it is judged that the direction of 
the lines is the lateral direction.. As shown in Fig 1 38, 

^5 there is a method whereby projections (histograms) in 
the vertical and lateral directions of a document are de- 
tected and the direction is judged by checking a degree 
of delimiter, orthe like. According to the above methods, 
the direction is fairly arbitrarily decided. According to 

^0 such methods, it is impossible to cope with documents 
of many kinds of formats as shown in Figs, 11 A to 11 D. 
For example, a condition such that the separator in the 
lateral direction exists in the document, the document is 
laterally written, or the like is needed. As for the docu- 

5S ment which doesn't satisfy such conditions, the direction 
of the original cannot be automatically discriminated. 

According to the invention, attention is paid to an 
idea such that a factor which most accurately expresses 



2 

BNSDOCiD: <EP 0725359A1_L> 



3 



EP 0 725 358 A"! 



4 



the direction of a document is a character For example, 
in one embodiment a character recognition is performed 
from four directions of 0"*, 90^ 180'' and 270**, with re- 
spect to a plurality of characters among a plurality of 
character areas for inputted document image data, an 
average value of precision values (reliabilities) of the 
character recognition per direction of a plurality of char- 
acters is obtained for every character area, a mean val- 
ue in the whole character area of the average value per 
direction in each character area obtained is calculated, 
and the direction in which the mean value is largest is 
discriminated as a document direction, so that the direc- 
tion of the document can be automatically judged at a 
high precision. 

By using such a method, the direction of the docu- 
ment can be automatically discriminated at a high pre- 
cision irrespective ot the format of the document. 

According to the invention, In the case where the 
document is not placed In the proper direction, by cor- 
recting the direction of the document by rotating the im- 
age information in accordance with the information in 
the proper direction which was automatically judged, a 
burden of the operation of the operator is eliminated and 
the image inputting operations up to at least the char- 
acter recognition can be automated. 

According to the invention, by automatically judging 
the direction of the original and performing the character 
recognition in the proper direction, a high recognition ra- 
tio can be obtained 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig„ 1 is a system constructional diagram of a data 
processing system to which a document direction 
automatic discriminating apparatus and a-^docu- 
ment direction automatic correcting appardtus ac- 
cording to an embodiment of the invention are ap- 
plied; 

Fig, 2 is a flowchart showing processes for automat- 
ic discrimination and correction of a document di- 
rection and character recognition according to the 
embodiment of the invention; 
Figs, 3A to 3D are diagrams showing area separa- 
tion states; 

Figs. 4A to 4D are explanatory diagrams for explain- 
ing processing steps of the character recognizing 
process; 

Figs, 5A to 5C are explanatory diagrams for explain- 
ing the document (character) direction discriminat- 
ing process; 

Figs. 6 A to 60 are diagrams showing data formats 
of area separation and character recognition infor- 
mation, 

Fig„ 7 is an explanatory diagram for explaining an- 
other embodiment; 

Fig, 8 is an explanatory diagram for explaining an- 
other embodiment different from Fig 7; 
Fig , 9 is a flowchart showing processes for automat- 
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ic discrimination and correction of a document di- 
rection and character recognition according to an- 
other embodiment of Fig, 8; 

Figs, 1 0A to 1 0D are diagrams showing an example 
of a character recognition result in each direction 
for the same character; 

Figs. 11 A to 11 D are diagrams showing examples 
of formats such as a character direction and the like 
of various kinds of documents; 
Figs, 1 2A and 12B are explanatory diagrams for ex- 
plaining a problem in case of displaying a document 
on a monitor; 

Figs. 1 3A and 1 38 are explanatory diagrams for ex- 
plaining conventional document direction discrimi- 
nating methods; 

Figs.. 14A and 148 are diagrams for explaining a 
character extraction of an English document; and 
Figs: ISA to 15C are diagrams showing results in 
the case where a character "T" is recognized in four 
directions. 



DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 



2^ Embodiments of the present invention will now be 
described in detail hereinbelow with reference to the 
drawings. 

Fig, 1 is a system constructional diagram of a data 
processing system to which a document direction auto- 

30 matic discriminating apparatus and a document direc- 
tion automatic correcting apparatus according to the 
embodiment of the invention are applied. The system 
has an image input unit and an image processing unit 
and is realized using either one of a scanner, a copying 

35 apparatus, an intelligent input device and a sole input 
device which is connected to a computer through an in- 
terface to be made intelligent^ 

A scanner 1 is means for optically reading a docu- 
ment original and photoelectricaiiy converting and input- 

40 ting as digital image data By attaching an auto feeder 
to the scanner 1 , a plurality of originals can be continu- 
ously inputted., A CPU/memory 2 is provided for execut- 
ing various kinds of controls and temporarily storing im- 
age data. 

By paying attention to a point that a factor which 
most accurately expresses the direction of a document 
is a character, a character recognizer/direction discrim- 
inator 3 performs a character recognition to several 
kinds of character areas in the document from the direc- 

so tions of 0^, 90^ 180^ and 270'' and sets the direction 
in which the precision is highest among the precisions 
of the character recognition in the respective directions 
(reliabilities of the character recognition: distances for a 
feature distribution of a character) to the document di- 

^5 rection. 

An area separator 4 is a block to execute processes 
for separating a character area, a figure area, a natural 
image area, a table area, and the like into rectangular 
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areas fronrvthe document image data and adding an at- 
tribute (character area or the like) of each area as a pre- 
process tor perfornning the character recognizing and di- 
rection discriminating processes by the character rec- 
ognizer/direction discriminator 3 s 

A storage 5 is constructed by, for example, a hard 
disk, a magnetooptic disk, or the like and is used to pre- 
serve various processing results (image data, area sep- 
aration result, character recognition result, etc ) An in- 
terface (l/F) 6 is constructed by an SCSI, an RS232C. io 
or the like and is provided to transmit data to the outside, 
A computer 7 obtains and uses information through the 
l/F 6 or data from a movable storage such as a magne- 
tooptic disk or the llke„ For the image data inputted from 
the scanner 1 , a printer 8 prints and outputs image data 
or the like which was subjected to various kinds of proc- 
esses on the basis of area separation information and 
character recognition information. 

An outline of the processes for the document direc- 
tion automatic discrimination and correction and the 20 
character recognition in the embodiment will now be de- 
scribed in accordance with a flowchart of Fig. 2. 

The image data (binary image or multi-level image) 
which was inputted by the scanner 1 and stored in the 
memory 2 is first separated into rectangular areas by 
the area separator 4 for every attribute such as charac- 
ter area, figure area, natural image area, table area, etc. 
(steps 31, S2). In this instance, area information sur- 
rounded by a rectangle is actually formed The area in- 
formation is stored into the memory 2 by making coor- 30 
dinate data indicative of the area and attribute (charac- 
ter, figure, natural image, table, etc.) information of the 
area correspond to each other. 

Subsequently, the area information in which the at- 
tribute information indicative of the character area 
among the attributes is made correspond is extracted 
from the memory 2 (step S3). The character area relates 
to a sentence area, a title area, a character in a table, a 
caption area of a diagram, or the like and is an image 
area including at least one character pattern,, For exam- 40 
pie, in case of documents shown in Figs, 3A and 3C, 
rectangle information of character areas as shown in 
Figs„ 3B and 3D is discriminated in step S2 and is ex- 
tracted in step S3. The document direction is judged by 
using a few blocks in those rectangle information (step ^5 
S4). Thus, when the document direction is the proper 
direction, the character recognizing process is also sub- 
sequently executed to the other character areas in the 
images stored in the memory 2 (step S7). 

On the other hand, when the document direction is 
the improper direction, a correction to rotate the image 
data in the correct direction is executed (step S5)„ The 
area separating process similar to step S2 is executed 
to the rotated image and a process such that the area 
separation information which was extracted from the im- 55 
age before the direction is corrected and which has al- 
ready been stored in the memory 2 is corrected to the 
area separation information extracted from the image 



whose direction was corrected is executed (step S6),, In 
this process, a difference between the area separation 
information in association with the image rotation is cor- 
rected. As such a correcting method, there is a method 
of again executing the area separating process to the 
rotated image data obtained in step S5 There is also 
another method of performing an address conversion to 
the area separation information obtained in step S2 and 
stored in the memory 2, Since the area separating proc- 
ess generally assumes that an image as a target to be 
processed is placed in the proper direction, in many cas- 
es, the result of the area separating process which was 
performed at the initial stage and the result of the area 
separating process which was executed for the rotated 
image data are different It is, accordingly, desirable to 
use the former method. 

The processing routine advances to step 87 and a 
character area block in the rotated image data is sub- 
jected to the character recognition by the character rec- 
ognition processing system.. Thus, the area separation 
information and character recognition information are fi- 
nally obtained with respect to both cases where there is 
no rotation and where there is a rotation (step SB)., 

The processing result is transmitted to the computer 
7 through the l/F 6. A text of the character recognition 
result is displayed by a display unit of the computer and 
is used for an application software or the like of the filing 
on the computer 7„ In the system to transfer the process- 
ing result to the storage 5. the result is used for a method 
which is used like a batch processing such that the im- 
age information is continuously inputted and the infor- 
mation is subsequently read out in a lump,. Further, in 
the system totransferthe processing resuitto the printer 
8, so long as the printer 8 has a function to interpret a 
page describing language, the processing result is used 
when a document is reconstructed or a fair copy is made 
by an inverse PDL (method of forming a page describing 
language by the image data) by the character recogniz- 
ing process and area separating process. 

In the case where a plurality of images are inputted 
in a lump, the document direction is discriminated with 
respect to each document, the document direction is 
stored into the memory 2 in correspondence to the page 
ID information of the image, and the image of each doc- 
ument can be also subjected to the character recogni- 
tion in accordance with the page ID information in step 
SB. 

A method of discriminating the document direction 
by using the character recognizing process will now be 
described. 

[Area separating process] 

The area separating process which is executed by 
the area separator 4 in step S2 will now be described. 

Black pixels of the document image data stored in 
the memory 2 are detected, the area in which the black 
pixels exist is extracted by an outline tracing or labelling 
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method, and area information as a group of images is 
formed Subsequently, a black pixel density in the rec- 
tangular area, the presence or absence of the adjacent 
rectangular block, a ratio between the vertical and lat- 
eral dimensions of the rectangle, or the like is used as 
a judgment reference, thereby discriminating the at- 
tribute of the image such as character area (title, body, 
caption, etc.), figure area, natural image area, table ar- 
ea, or the like with respect to each image area„ 

[Character recognizing process] 

The character recognizing process which is execut- 
ed by the character recognizer 3 for the document di- 
rection discriminating process (S4) and the conversion 
of the image into the text (S7) will now be described.. 

As a method for the character recognizing process, 
there is a feature vector extracting and comparing meth- 
od, For example, as shown in Fig 4A, it is now assumed 
that a character area 41 including a character 40 was 
discriminated by the area separating process and was 
specified as a target of the character recognizing- proc- 
ess. As a first stage, a character extracting process is 
executed with respect to the character area 41 (refer to 
Fig., 4B). The character extracting process is a process 
to extract a rectangle of one character and the rectangle 
is obtained by detecting a state of continuity of the black 
pixels or by getting a histogram. According to the proc- 
ess at the first stage, total six character patterns of char- 
acters 42 to 47 are extracted from the character area 
41 . As a second stage, one character is divided into (m 
X n) [for example, 54 X 64] pixel blocks and those 
blocks are extracted (refer to Fig. 40). The distribution 
direction of the black pixels is extracted (direction vector 
information: refer to Fig. 4D) by using a window of (3 x 
3) pixels among them„ 

Fig. 4D shows an example of a part of the direction 
vectocinformation. By shifting the window of (3 x 3) pix- 
els, tens of direction vector information are obtained. 
The feature vector information becomes a feature of the 
input character. Vector information similar to it has been 
also stored as standard data in a character recognition 
dictionary. The feature vector extracted from the input 
character and the contents of the character recognition 
dictionary which has previously been stored are com- 
pared. Candidate characters are sequentially extracted 
from the dictionary in accordance with the order from 
the character whose feature is closest to the feature vec- 
tor extracted from the input character In this case, the 
first candidate, second candidate, third candidate, 
are extracted in accordance with the order from the 
characters whose features are close to the feature vec- 
tor A similarity indicative of a closeness of the feature 
to the feature vector, namely, closeness of the distance 
to the character becomes a numerical value called a re- 
liability (precision) of the result of the character recog- 
nition. 



[Character direction discriminating process] 

In this way, the reliability of the result of each char- 
acter recognition is obtained. Each character direction 
5 discriminating process based on the reliability will now 
be described with respect to an example in case of per- 
forming the character direction discriminating process 
as for the character area 41 shown in Fig, 5A, 

First, explanation will now be made with respect to 
processes such that the character 40 as a first character 
in the character area 41 is recognized in four directions 
and the direction of the character pattern 40 of the first 
cliaracter is discriminated from the reliability of the char- 
acter-recognition result derived in each direction. 

Fig. 5A shows a sentence in the proper direction. 
Fig. 5B shows a sentence rotated by 270^. Attention is 
now paid to the character pattern 40 of the first character 
which is first extracted from the character area. In case 
of discriminating the character direction, as shown in 
Fjg. 5C, the character recognition is executed with re- 
spect to one character 40 from four directions of 0"*, 90", 
180°, and 270"*, respectively It is sufficient to set each 
rotational angle by changing a reading method of the 
image of the character rectangle from the memory 2 
when the character recognition is performed,, There is 
no need to particularly rotate the original 

As shown in Fig, 5C, the character recognition re- 
sults at the respective rotational angles are different 
from each other The temporary character recognition 
results and reliabilities for explanation are shown in Fig. 
5C and the actual results and reliabilities do not always 
coincide them. 

In Fig 5C, when the character recognition is exe- 
cuted from O'', the innage is correctly recognized as a 
character 50 arid the reliability is also equal to a high 
value of 0.90. In case of performing the character rec- 
ognition from the direction rotated by 90**, the image is 
erroneously recognized as a character 51 and the reli- 
ability also decreases to 0.40, The reason why the er- 
roneous recognition occurs and the reliability also dete- 
riorates is because the character recognition is per- 
formed on the basis of the feature vector in case of see- 
ing from the rotated direction. Similarly when the char- 
acter recognition is executed from the rotated directions 
of 180"* and 270"* as well, the erroneous recognition oc- 
curs and the reliability deteriorates. As the character is 
complicated, the difference between the reliabilities for 
the directions of the character recognition typically ap- 
pears. 

As for the result of Fig. 5C, since the reliability of 0° 
is the highest, it is judged that a possibility such that the 
document faces In the proper direction is high. To further 
improve the precision of the discrimination about the 
character direction, the character recognition is similarly 
executed from four directions with respect to a plurality 
of characters in the same block. For example, in case 
of the character area 41 shown in Fig 4A: the character 
recognition is performed from four directions with re- 



15 



20 



2B 



30 



35 



40 



45 



SO 



5 



BNSDOCiD: 0725359A1 J„> 



9 



BP 0 725 3B9 A1 



10 



spect to each character of not only the character 42 but 
also the characters 43 to 47 Further, in the case where 
the character direction is judged in only one block, since 
there is a possibility such that the character direction is 
erroneously judged with respect to a special character s 
train, a similar character recognition can be also execut- 
ed with respect to a plurality of blocks, 

It is also possible to construct in a manner such that 
with regard to each block, the average values of the re~ 
iiabilities for every four directions of each of the recog- io 
nition target characters in the relevant block are ob- 
tained, the direction in which the highest average value 
among them is derived is judged as a character direction 
of the block, and such a direction is decided as a docu- 
nnent direction of the input image Further, the character 
pattern of the character direction discrimination is se- 
lected from a plurality of character areas, the mean val- 
ues for the average values of the reliabiiities for every 
tour directions in each block is obtained, and the direc- 
tion in which the mean value is the highest among them 20 
is recognized as a character direction (document direc- 
tion). 

As mentioned above, by recognizing the character 
direction on the basis of a plurality of characters in the 
same block and, further, on the basis of the reliabilities 
of a plurality of characters in the same block without rec- 
ognizing the character direction by the reliability of only 
one character the character (document) direction can 
be discriminated at a high precision It will be obviously 
understood that even if the character direction is dis- 30 
criminated by the reliability of only one character or is 
judged by the reliabilities of a plurality of characters in 
the same block, the character direction can be discrim- 
inated at a higher precision than the conventional one. 

When the discrimination result of the character di~ 35 
rection (document direction) indicates the direction oth- 
erthan the proper direction, the original image stored in 
the memory 2 is rotated so that the character direction 
is set to the proper direction. The rotation can be easily 
executed by the well-known technique by using the ^0 
CPU/memory 2 in Fig., 1 and its description is omitted 
here„ 

By the processes as mentioned above, original im- 
age data shown in Fig, 6A, area separation data shown 
In Fig„ 6B, and character recognition information shown 
in Fig. 60 can be obtained. Those information is used 
by an application such as electronic filing, document 
modification, DTP, or the like as mentioned above. 

A format of the area separation data which is ob- 
tained by the area process in step 32 and is stored in so 
the memor/ 2 is constructed by "header" indicative of 
the area separation data and identifiers "recti" to "rect4" 
of the separated areas as shown in Fig 68. Information 
of each area (block) classified by those identifiers is con- 
structed by: a number "order" of the block; an attribute 55 
(character area, figure area, etc. ) "att" of the block; co- 
ordinate values "x1 " and "y1 " at the left upper position 
of the block; a width "w" of the block; a heigh! "h" of the 



block; "direction" indicating that the document is written 
in the vertical or lateral direction; "Self ID" as an ID of the 
relevant block; "upperlD" as an ID of a master block in- 
cluding the relevant block, an attribute "upperAtt" of the 
master block; and a reserved area "resen/e"„ 

As shown in Fig, 6C, the character recognition in- 
formation has "header" indicative of the character rec- 
ognition information and is constructed by combination 
information of character recognition information "OCR1 " 
or the like regarding a single character such as "char- 
acter 40" or the like and "blk header" corresponding to 
the above identifier "recti" or the like indicative of the 
block including the relevant character. 

Each character recognition information such as 
"OCR1" or the like is constructed by: "type" indicative of 
a character or a blank; first to fifth candidate characters 
"character!" to "characters" according to The reliability of 
the character recognition mentioned above; extracting 
positions "x1" and "y1 " of the character; a width "w" of 
the character, a height "h" of the character, and a re- 
served area "reserve". 

According to the embodiment, in the case where the 
document doesn't face the proper direction as a result 
of the discrimination of the document direction and the 
image has to be rotated, the whole original image data 
is rotated and the character recognition is executed. 
However, the character recognition can be also per- 
formed without rotating the original image data 

In this case, when the rectangle information of the 
character area is obtained, as shown in Fig. 7, it is suf- 
ficient that only the character rectangle data is rotated 
and copied into another memory and the character rec- 
ognition is executed for the rectangle image data,. 

As shown in Fig,. 8, it is also possible that a charac- 
ter extraction is executed for the character rectangular 
area without rotating the image data and when each 
character is read out, it is read out in a state in which 
the reading direction is rotated, and the character rec- 
ognition is performed (refer to step S9 in Fig. 9)„ Since 
the other steps in a flowchart of Fig,, 9 are almost similar 
to those mentioned in Fig, 1 , they are designated by the 
same step numbers and their descriptions are omitted 
here,. 

In the flowchart of Fig. 2, all of the processes from 
the input of the image to the process to form the text of 
the image data by the character recognition have been 
described as a series of processes. However, the image 
data in which it is judged that the document direction is 
the proper direction as a result of the discrimination of 
the document direction In S4 and the image data as a 
result in which the image was rotated (S5) and corrected 
to the proper direction are outputted by display means 
(not shown) or the printer 8, Thus, the user can auto- 
matically obtain the image data in the proper direction 
and the subsequent working efficiency can be raised. 

The invention is not limited to each of the above em- 
bodiments but can be also applied to, for example, doc- 
uments written by other languages instead of limiting to 
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only Japanese Particuiarly, in case of the docunnent 
written in English or the like, since there is no docunnent 
written verticaliy, the document direction can be discrinn- 
inated at a higher precision„ 

The document direction discriminating process in s 
the case where the original as a target of the judgment 
about the document direction is written in English wiii 
now be described with reference to Figs„ 14A, 14B, and 
15A to 15C, 

An Image 1400 inputted in S1 is stored into the io 
memory 2 and four image areas 1401 to 1404 are ex- 
tracted by the area separator 4 (S2) It is judged that 
three areas 1401 to 1403 among the extracted image 
areas are character areas and one area 1 404 is a figure 
area from the feature of the image. is 

In the detection of the character area in S3, the area 
information of the three image areas 1 40 1 to 1 403 which 
were judged as character areas in S2 is read out from 
the memory 2 and the process to discriminate the doc- 
ument direction from the image data of each area is ex- 
ecuted (S3). 

For example, when the character patterns are ex- 
tracted from the image data in the character area 1401, 
five characters 1411 to 1415 are extracted as shown in 
Fig„ 14B, Among them, the results obtained by recog- 25 
nizing the first character "T" from four directions are as 
shown in Fig,. 15C. Correctly, when the character 1411 
of "T" is recognized from 0^, the result such that the rec- 
ognized character (first candidate character) is "T" and 
the reliability is equal to 0.95 is obtained. When the char- 30 
acter 1411 is recognized from 90^ the result such that 
the recognized character Is and the reliability is 
equal to 0.40 is obtained,. Similarly, in case of 180°, the 
recognized character is "1 " and the reliability is equal to 
0.50„ In case of 270'', the recognized character is "4--" 35 
and the reliability is equal to 0.40. 

From the above results, the recognized character 
"T" with the highest reliability corresponds to the correct 
recognition result of the character 1411 . The rotational 
angle 0** at which the correct recognition result is ob- 40 
tained is judged to be the direction of the character 1 41T 

It is also possible to discriminate that the direction 
judged here is set to the document direction of the image 
1400. 

Or, in order to discriminate the document direction ^5 
at a higher precision, It is also possible to construct in a 
manner such that the character recognition is performed 
with respect to all of the characters 1411 to 141 5 includ- 
ed in the character area 1401 selected from ail of the 
character areas (1 401 to 1 403) extracted in step S2 and ^0 
the average value of the reliabilities is obtained in each 
direction and the direction in which the average value is 
the highest among them is judged as a direction of the 
document. It is also possible to selectively perform the 
character recognition to, for example, only the first and 
fifth characters in accordance with a predetermined or- 
der instead of executing the character recognition with 
regard to all of the characters 1411 to 1415, In this case, 



the characters "T" and "E" are recognized. 

Or. in order to discriminate the document direction 
at a further high precision, it is also possible to select 
and recognize the characters from all of the character 
areas extracted in S2 and to discriminate the document 
direction by using the recognition results, in this case, 
there is used a method whereby the average value of 
the reliabilities of all of the recognition results is obtained 
for each direction and the direction in which the average 
value is the largest among them is discriminated as a 
document direction. Or, there is used a method whereby 
the average value of the reliabilities for each direction 
is obtained every area and the mean' value obtained of 
the average values of each area is obtained with respect 
to ail of the areas and the direction in which the mean 
value is the highest among them is discriminated as a 
document direction. 

The characters to be recognized can be selected at 
random or can be sequentially selected in accordance 
with a predetermined order as mentioned above. For ex- 
ample, now assuming that the first and fifth characters 
are selected In a manner similar to the above, "T" and 
"E" are selected from the character area 1401, "I" and 
"T" are selected from the character area 1402, and "D" 
and "M" are selected from the character area 1403 The 
document direction of the image 1400 is discriminated" 
in accordance with the results df the character recogni- 
tion of those six characters. 

In case of English, since only the documents written 
in the lateral direction exist, the direction in which the 
image in the character area extracted by the area sep- 
arating process is placed laterally assumes 0° in accord- 
ance with the shape of the extracted character area„ 
Since the documents of 90^^ and 270" cannot exist, the 
character recognizing process is omitted. The character 
recognrtion is peilormed with respect to only 0** and 
180** and the document direction can be judged by 
checking which reliability is higher 

Since the character recognizing process can be re- 
duced by the above processing procedure, the process- 
ing time for document discrimination can be reduced. 



Claims 

1. An image processing apparatus characterized by 
comprising: 

image input means for photoelectrically con- 
verting a document Image which was optically 
read and for inputting as image data; 
a character recognition dictionary in which fea- 
tures of characters have been stored; and 
discriminating means for performing a charac- 
ter recognition from a plurality of directions with 
respect to character data in the image data in- 
putted by said image input means with refer- 
ence to said character recognition dictionary, 
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thereby discriminating a document direction on 
the basis of the results of said character recog- 
nition, 

2. An apparatus according to claim 1 , characterized in 
that said discriminating means performs the char- 
acter recognition from a plurality of directions with 
respect to the character data in said inputted image 
data with reference to said character recognition 
dictionary, thereby discriminating a character rec- 
ognizing direction according to a character in which 
a precision value of the character recognition is 
highest as a document direction 

3„ An apparatus according to claim 1 , characterized in 
that said discriminating means extracts a character 
area including a plurality of character patterns from 
said inputted image data and performs the charac- 
ter recognition from a plurality of directions with re- 
spect to each of the plurality of character patterns 
in said extracted character area, thereby discrimi- 
nating the document direction on the basis of the 
character recognition results obtained from said 
plurality of character patterns „ 

4. An apparatus according to claim 1 , characterized in 
that said discriminating means extracts a character 
area from said inputted image data and performs 
the character recognition from a plurality of direc- 
tions with respect to each of a plurality of character 
data in said extracted character area, thereby dis- 
criminating a direction in which the average value 
for each direction of precision values of the charac- 
ter recognition of a plurality of characters is largest 
as a document direction., 

5. An apparatus according to claim 1 , characterized in 
that said discriminating means extracts a plurality 
of character areas from said inputted image data, 
performs the character recognition from a plurality 
of directions with respect to each of a plurality of 
character data in said plurality of extracted charac- 
ter areas, and further obtains average values of pre- 
cision values of the character recognition for each 
direction of a plurality of characters with respect to 
each of the character area, and obtains a mean val- 
ue of the average values for each of said directions 
In each of the character areas, thereby discriminat- 
ing a direction in which the mean value of said av- 
erage values is largest as a document direction. 

6. An apparatus according to claim 1, characterized 
by further having character recognizing means for 
character recognizing image information in a proper 
direction in accordance with the direction discrimi- 
nated by said discriminating means„ 

7. An apparatus according to claim 1 , characterized in 



that said image input means is a scanner 

8. An apparatus according to claim 1 , characterized in 
that said image input means inputs a plurality of 

5 document images, and 

said discriminating means discriminates the 
document direction with respect to each of said 
plurality of document images 

10 

9. An apparatus according to claim 1 , characterized 
by further having correcting means for correcting 
the image data so that said character data is set to 
a proper direction on the basis of the results dis- 

is criminated by said discriminating means, 

10. An apparatus according to claim 9, characterized 
by further having character recognizing means for 
character recognizing said image data corrected by 

20 said correcting means,, 

11. An apparatus according to claim 9, characterized 
by further having display means for displaying said 
image data corrected by said correcting means, 

2S 

12. An apparatus according to claim 9, characterized 
by further having printing means for printing said im- 
age data corrected by said correcting means,, 

30 13. An apparatus according to claim 6, characterized 
by further having display means for displaying the 
results recognized by said character recognizing 
means „ 

3S 14, An apparatus according to claim 6, characterized 
by further having printing means lor displaying the 
results recognized by said character recognizing 
means 

40 15. An image processing method characterized by 
comprising the steps of: 

photoeiectrically converting a document image 
which was optically read and inputting as image 

45 data; 

performing a character recognition from a plu- 
rality of directions with respect to character da- 
ta in said inputted image data with reference to 
a character recognition dictionary in which fea- 

so tures of characters have been stored; and 

discriminating the document direction on the 
basis of results of said character recognition 

16. A method according to claim 15, characterized in 
S5 that in said discriminating step of said document di- 
rection, the character recognition is performed from 
a plurality of directions with respect to character da- 
ta in said inputted image data with reference to said 
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character recognition dictionary, and the character 
recognizing direction regarding a character in whicli 
a precisian value of the character recognition is 
highest is discriminated as a document direction. 

17. A method according to claim 15, characterized in 
that in said discriminating step of said document di- 
rection, a character area including a pluinlity of 
character patterns is extracted from said inputted 
image data, the character recognition is performed 
from a plurality of directions with respect to each of 
said plurality of character patterns in said extracted 
character area, and the document direction is dis- 
criminated on the basis of the character recognition 
results obtained from said plurality of character pat- 
terns, 

18. A method according to claim 15, characterized in 
that in said discriminating step of said document di- 
rection, a character area is extracted from said in- 
putted image data, the character recognition is per- 
formed from a plurality of directions with respect to 
each of a plurality of character data in said extracted 
character area, and a direction in which an average 
value for each direction of precision values of the 
character recognrtion of a plurality of characters is 
largest is discriminated as a document direction, 

19. A method according to claim 15, characterized in 
that in said discriminating step of said document di- 
rection, a plurality of character areas are extracted 
from said inputted image data, the character recog- 
nition is performed from a plurality of directions with 
respect to each of a plurality of character data in 
said plurality of extracted character areas, average 
values of precision values of the character recogni- 
tion for each direction of a plurality of characters are 
obtained for each of said character areas, a mean 
value of the average values for each of said direc- 
tions in each of said character areas is obtained, 
and a direction in which the mean value of said av- 
erage values is largest is discriminated as a docu- 
ment direct ion « 

20. A method according to claim 15, characterized in 
that image information in a proper direction is char- 
acter recognized in accordance with said discrimi- 
nated direction, 

21. A method according to claim 15, characterized in 
that said image data is inputted by a scanner 

22. A method according to claim 15, characterized in 
that 

image data of a plurality of documents is input- 
ted, and 

in said discriminating step of said document di- 



rection, the document direction is discriminated 
with respect to each of the image data of said 
plurality of documents 

s 23. A method according to claim 15, characterized in 
that the image data is corrected so that said char- 
acter data is set to a proper direction on the basis 
of said discriminated results, 

10 24. A method according to claim 23, characterized in 
that said corrected image data is character recog- 
nized,, 

25. A method according to claim 23, characterized in 
is that said corrected image data is displayed by dis- 
play means, 

26. A method according to claim 23, characterized in 
that said corrected image data is printed by printing 

20 means, 

27. A method according to claim 24, characterized in 
that said character recognized results are displayed 
by display means. 

25 

28. A method according to claim 24, characterized in 
that said character recognized results are printed 
by printing means, 

30 29. A method or apparatus for character recognition 
having a pre-process of extracting a plurality of 
sample characters from an input image or part 
thereof, performing plural recognitions of each sam- 
ple character assuming different orientations and 

35 determining orientation for recognition processing 
of the whole image or image part as the orientation 
having the highest degree of success of recognition 
of the sample characters. 

40 30. A method or apparatus for character recognition 
wherein orientation of an input image is determined 
prior to recognition by identifying using some char- 
acters extracted from the image an orientation 
which maximises the certainty of recognition. 

45 

31. A method or apparatus having the features of any 
combination of the preceding claims. 
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FIG. 6A FIG. 6B FIG. 6C 
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